Streaming of digital data to a portable device

ABSTRACT

The present invention achieves wireless delivery of media streams to portable devices for providing media information related to a show or attraction for the purposes of captioning, language translation, and assistive listening. As a person moves about a venue, the device receives infrared or radio signals from transmitters at venue locations. The device decodes each signal as venue location, event time, or device trigger for presentation on the device. The device memory contains audio, text, graphics and device playback may contain one or many of these stored contents. The stored content is synchronized with the user&#39;s location, time of travel, time of day, time of a show, or an event in a show and may be in one or many languages. Presentation of stored content on the device may also be altered by user preference, stored logic constraints, automatic learning of user behavior, or user interaction with other devices.

RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application Ser.No. 60/421,255 entitled “Streaming of Digital Data to a PortableDevice”, filed on Oct. 25, 2002 which is incorporated herein byreference in its entirety.

BACKGROUND

1. Field of the Invention

The present invention relates generally to the use of infrared and/orradio frequencies to stream digital data to portable devices in a massaudience or in designated broadcast area.

2. Description of the Background Art

There are several circumstances which may interfere with a person'sability to hear, and thereby interfere with the presentation ofinformation to the listener. For example, persons with hearing loss maymiss narratives, sound effects, music and other sound material relatedto the presentation, and messages in live performances, films,television and special events. Persons who do not speak the language orlanguages used in the presentation may miss narratives and other relatedmessages that are presented. The language barrier prevents many peoplefrom different cultures and languages from understanding, participatingor interacting with the information being presented. Backgroundenvironmental noise may also affect a person's ability to hear, andthereby diminish the effect of the presentation. Additionally,presentations often could be made more effective if they included amethod to provide interactivity between the audience and thepresenter(s).

Captioning systems have been used in many venues including museums,theaters and other auditoriums to provide foreign language translationor captioning for the hearing impaired. These systems are either 1)“open captioning” on a projected surface or large adjacent display areawhere the entire audience can see the captioning; 2) reflectivecaptioning using a transparent but reflective panel to display the textfrom a rear projection while allowing the viewer to see the display orperformance through the panel; or, 3) hard-wired displays in the back ofthe seat in front of the viewer.

Wireless streaming systems also exist. These consist of modulated analogand digitized audio broadcasting systems such as assistive listeningsystems and wireless headphones. Also short-range (often 1 meter orless) digital transceivers are typically used to exchange data betweencomputers or between computers and peripherals. These systems do notsynchronize multiple units.

SUMMARY OF THE INVENTION

It is therefore an object of this invention to provide a method ofpresenting random and/or synchronized information such as narratives,translations, interactive games, control signal commands or other showrelated messages, to patrons of shows, movie theaters, exhibit halls,auditoriums and/or designated areas through an unobtrusive device.

The present invention makes use of infrared (IR) emission and/or radiofrequency (RF) transmission to stream data to a portable device forclosed captioning, language translation for multi-cultural languagegroups, previews, games, control of devices and/or similar applications.The portable captioning device could be carried by a person and/or thedevice could be placed in a fixture for hands free use. Before or at thetime of the start of a presentation or presentations, the IR/RF systemwill start the transmission of data to the portable units. Thetransmission will synchronize the portable device with the presentationor presentations for captioning, language translation, previews, games,control of devices and/or similar applications. The portable devicemight be based on existing technologies such as mobile phone, personaldigital assistant (PDA) or a combination of both mobile phone and PDA, acustom designed device specifically for this application, or aninteractive device. This system can be combined with an existing audiostream for the hearing impaired, descriptions for the blind and/orlanguage translation. For example, the invention may provide forinfrared streaming for assistive listening systems.

Possible user device implementations include a custom portable textdisplay terminal having a receiver, display controller,micro-controller, small amount of memory, and power source. Similarly, acustom audio unit having a receiver, digital-to-analog converter, audioamplifier, and speaker, could also be used.

Other devices include talking or motorized toys and games capable ofreceiving content appropriate to a presentation. Such devices may alsoallow interaction with nearby units. Special effects units having somecombination of lights, audio effects, or animations could also be used.These units could be held by the user or mounted in the presentationspace as a remotely controlled device. Still other devices include adigital map or way-finding unit, messaging system that allows the userto broadcast and receive data to and from nearby units, or useridentification unit that broadcasts user identity, preferences, orlocation to nearby units.

The present invention provides several benefits over prior art systems.Prior art systems are not aware of user preferences, such as the user'slanguage, gender, age, etc. Prior art wireless audio systems suffer fromsignal loss and deterioration of the transmitted audio when the usermoves relative to the infrared transmitter. Existing digital wirelessaudio systems do not have provisions for mixing text or control data inthe audio data stream.

The present invention provides for a combination of features not foundin prior art wireless data communication systems. For example, thesystem and method of the present invention allows multiple users'devices to simultaneously receive identical data and remain synchronizedwith the presentation(s). User devices are able to receive differenttypes of data, such as audio, animation control, text, etc. during asingle transmission. User devices are furthermore able to interact withthe presentation, interact with the presenters, or interact with eachother.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view of a system of the invention utilizing IRsignals.

FIG. 2 is a schematic view of a system of the invention utilizing RFsignals.

FIG. 3 is a schematic view of a portable display device receiving an IRsignal.

FIG. 4 is a schematic view of an embodiment of the present inventionutilizing radio frequency (RF) signals.

FIG. 5 is an exemplary depiction of a data packet stream delivered by ahigh speed transmitter.

FIG. 6 is an exemplary embodiment of the portable device in accordancewith the present invention.

FIGS. 7 a-c is a flow diagram in accordance with the teachings of thepresent invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Reference will now be made in detail to an exemplary embodiment of thepresent invention, an example which is illustrated in the accompanyingdrawings. It is to be understood that other embodiments may be utilizedand structural and functional changes may be made without departing fromthe respective scope of the present invention.

The hardware requirements of the preferred embodiment of this system aredepicted in FIGS. 1 and 2. A time code reader 10, which in thisembodiment is an optical time code reader, is capable of reading timecodes from a show device 12 such as a film projector, show controlcomputer or another media source. The media source may include analogand/or digital content and could be audio, video, and other information(e.g., force feedback parameters for tactile sensing). A centralprocessing unit (CPU) 14 receives the time code signal from the reader10 and synchronizes the content with the film and/or presentation, whichmay be a movie screen 15. Examples of content are text captioning,language translation, games and/or other related applications. Thecentral processing unit 14 has the capability to access and interpretthe content. The content can reside in the internal memory of thecentral processing unit 14 and/or as a removable memory media.

The CPU 14 will deliver the synchronized data to the infrared (IR)emitters 16, which are capable of delivering IR messages or control datato a portable device 18, which may be operated either in an indoor oroutdoor environment. Alternatively, a low-powered licensed and/ornon-licensed radio frequency (RF) system can also be used to deliver thesynchronized data to the portable device via an RF signal. FIG. 2depicts an RF receiver 24 and RF transmitter 22, which interact with aportable RF capable device 18 to achieve the same results as the IRsystem previously described.

The portable display device 18 has sufficient memory to allow storage ofall data to be presented. All required data may be transmitted to thedevice 18 in one session for display at predetermined times afterreceipt of a start signal, or the data may be transmitted in a pluralityof sessions for display in real time. The stored data may be displayedin its entirety, or a portion of the data may be displayed. The device18 includes infrared ports capable of receiving and/or emitting infraredmessages. Messages emitted from the portable display device 18 are readby one or more IR detectors 19. In an alternative embodiment, the device18 could also contain RF receiver and/or transmitter ports capable ofreceiving and transmitting RF messages. The portable device 18 willreceive the IR or RF signal and convert the signal to information thatcan be stored and/or displayed in sync with the presentation. The device18 may also contain the capability to receive and play audio such as forassistive listening and/or audio language translations, or programmaterial specific to the presentation.

The system could also recognize show/presentation start and/or endsignals. The system could then transmit random and/or synchronizedinformation to the patrons possessing a device. This will allow thepatrons to interact with the device while waiting for theshow/presentation to start or after the show/presentation has ended.

One possible application of the invention is depicted in FIG. 3. In thisapplication, time code information is read by the CPU 14, which thenaccesses on board content, and delivers the synchronized data to theinfrared emitters 16, which are in turn capable of delivering IRmessages. A portable device 24 receives the IR messages and converts theIR messages to presentable data. In this instance, the IR receiver 19 isa modification to an existing device such as a PDA (for example, a Palmtype device) and/or a pocket PC (for example, a Compaq iPAQ) that canstore and/or immediately display the data. The IR receiver takes the IRsignal from the emitter and translates it to an electronic signal forthe serial port 22 of the PDA and/or pocket PC. A terminal softwareprogram converts the electronic signal into data that is presented astext on the display screen 26.

This invention may be adapted for uses such as text captioning andlanguage translation in movie or live theaters, consumer products whichcan provide an interactive experience, and to provide a wireless linkfor control signals to equipment, devices or products which are used inpublic presentations.

Another application of the invention is depicted in FIG. 4. In thisapplication, a combination of RF transmitter 30 and IR emitters 31broadcast data to portable devices 33. The portable devices have both RFand IR receivers. The RF data stream contains encrypted caption text,digitized audio, and control data for one theater 34 or multiple venues.Each theater or venue has one or more IR emitters that send smallamounts of data to synchronize the portable device playback to userposition or the theater presentation. An emitter mounted outside onetheater's entrance may send an identifying code that the portable deviceuses to identify that theater. The device then extracts data from the RFstream for that theater's content, and loads the content into thedevice's memory. Content may include theater name, show times, showsynopses, as well as the text and audio for an upcoming show. When theuser enters the theater, the device receives IR synchronizing codes forthe current show, and the device displays caption text, or play audio,or otherwise synchronize device functions during the show. The IRsynchronizing codes are derived from the show media as describedearlier.

FIG. 5 illustrates an exemplary embodiment of the basic structure of thedata packet stream sent by the high-speed transmitters.

The figure shows a possible broadcast packet stream of theater showcontent for three theaters, along with data packets of ancillaryinformation such as theater show schedules and current date and time.

The broadcast packet stream contains small groups of data (packets) thatcontain a small amount of content that will be received by the userdevices and, if needed, will be loaded into the user devices localmemory. The logical state of each user device determines which packetsare loaded into a particular user device. For example, user devices thatare in theater A would receive and store content for theater A, butignore content for theaters B and C. All user devices would receive thecurrent time packet, and use the time data to reset the user deviceclock. Any user device in front of theater A, B, or C would receive andstore the theaters' schedule data. The example packets in the figureshow content broken into one-second fragments. So a captioning device intheater A would receive the caption text packet for the 1st second ofthe show, caption text packet for the 2nd second, etc. Each second ofcontent would be loaded into device memory for access during the show.

A large number of individual packet types are possible, each containingdifferent data such as caption text, audio content, video content,graphics, images, time, schedules, menus, geographical information, gamecontent, survey questions, advertising, warnings, alerts, etc.

Each data packet (refer to examples Caption Text, Audio Content, CurrentTime Packets in FIG. 5) always starts with a unique Header value thatsignifies the start of a new data packet.

The data packet also contains a Count value that is used to indicate theamount of following data in the packet. The Count is used to validatethe packet and calculate a checksum.

The Encryption data represents key and authentication information. This,and the Checksum, enables the user device to verify that following datain the packet is intended for the device and that the data has not beencorrupted. Corresponding key and authentication information are loadedinto the user device.

The Message Type value defines the content data in the packet. Based onthe Message Type, the user device will determine how to decode thepacket data, where to store the data, how and when to use the data. Forexample, warning text may be immediately displayed; show caption textmay be buffered in device memory until an IR synchronizing code triggersthat caption text display. Menu or schedule information may be bufferedin device memory until the user requests it.

The Address value designates an identifier for a receiving device. Thismay be a unique address, so the packet is intended for only one userdevice. Or, the address may specify a block of user devices, such as allcaptioning devices, or all game devices. Or, the Address may denote thepacket as a generic broadcast sent to all devices.

The optional Start Time value acts as a packet sequence number, but mayalso be used by the receiving user device to define when the packetcontent is used. As packets are received, the user device will buffercontent into its memory in sequence based on start time. As the userdevice receives synchronizing codes, it will pull content from thesequential memory or search memory for the content with appropriatestart time. Additional information such as Caption Text Data withFormatting codes, Digitized Audio content, and Current Time and Frameinformation are included in the Data Packet Stream.

there is a Checksum information that uses a combination of all thepreceding data to verify that the packet has arrived at its location inits entirety.

FIG. 6 illustrates an exemplary embodiment of a portable user device inaccordance with the present invention. All portable devices have thecommon elements of an IR receiver 50, RF receiver 60, IR transmitter 51,RF transmitter 61, processor 52, memory 54, and power source 56. Otherelements of the devices vary depending on intended applications. In thisexemplary embodiment, the portable device has both a display 53 fordisplaying text and/or graphics, in addition to an audio amplifier 57for playing sounds, which may be in the form of speech or musical tones.Memory 54 is where the data is stored on the device. The IR receiver 50receives a code from the wireless transmitter, and processor 52 analyzesthe code. The processor 52 interprets the code to determine how tohandle it. If the code is intended for that device, the device searchesits memory 54 for content corresponding to the code. The content is thendisplayed on display 53 and/or plays sound through the audio amplifier57. The portable device shown in FIG. 5 also shows user inputs 55, suchas pushbuttons, which may be used for interactive applications. Thehigh-speed RF transceiver system could be used to transfer content tothe portable device, while the IR tranceiver system may be used toprecisely aim location synchronizing codes to the portable device.

FIG. 7 is a flowchart that shows how received data packets are handledby the user device. After the device is powered on, it is capable ofreceiving data packets from both IR and RF receivers. The device alsomonitors user inputs that may come from pushbuttons, touch screen,speech input, assistive devices, etc. The device also periodically readsits clock, and searches its event database memory for preloaded eventsset to trigger at the current time.

Either receiver may recognize an incoming packet. Any packet thatcontains timing errors, corrupted data, cannot be authenticated, etc.will be ignored. A packet that has correct bit timing, correct header,can be authenticated, can be decrypted, and has a correct checksum willbe marked as valid.

A valid packet will be compared with the user device configuration, sothat only appropriate packets will be used. For example, a user devicethat displays caption text, but has no audio capability, may beconfigured for text only. That device would ignore packets containingaudio content.

The packet message type determines how the device will use the packetdata. The packet may contain content such as caption text, audio, orvideo that is buffered in the user device content database memory forfuture use. Other data, such as an urgent text alert message, may besent to the device display immediately. Data such as location and timesynchronizing codes may cause the device to search its content databasememory, extract the content, and present the content. Current time datamay cause the device to reset its internal clock. Event time data willbe placed in the device's content database memory, and will be acted onwhen the internal clock matches the event time. Special effect data maycause the user device to buzz, ring, animate, etc.

The user device may have some number of user inputs available. These maytake the form of pushbuttons, touch-screen soft buttons, touch-screenmenus, speech input, assistive device such as a breath-operated puffswitch, etc. The user may set preferences, such as points of interest,types of movies, age, educational level, etc. The device may use thesemanually entered preferences, along with learned preferences, to alterthe presentation of content. For example, a child's device may be set todisplay captions using a small English vocabulary. As the child carriesthe device from location to location in a museum, the device may also“learn”, or infer, the child's preference for a certain type of display.

The device may also learn a user's direction of travel, and thereforepredict or suggest future destinations. For example, after walking downa hallway towards North, the device may alert the user that the hallwaywill end, and a turn to the East or West is necessary. Furthermore, thelearned direction, along with any learned and manually entered userpreferences, may cause the device to deduce a preference for an exhibitthat is to the East of the hallway end.

The portable device may automatically infer user preferences from use ofthe device based on the current location of the user device, the timeelapsed at a location, path history (i.e. geographic walking history) oflocations visited, and the time elapsed between locations (i.e. walkingspeed).

User inputs may also be assigned to device hardware functions. Forexample, an input pushbutton may cause an IR transmitter to send a codeto an adjacent device. Such code may identify the user to another user,or identify the user to an automated device, network, etc. One code maycause a door to open, for example, or allow interaction with a gamesystem, etc.

The user device may present content based on internal logic and itsinternal clock, without receiving data packets. For example, textcaptions may appear at a predefined time of day to remind a user ofvenue show times, closing time, or remind the user of medication needed,etc. Further, such time events may be logically constrained by thelearned and manually entered user preferences. For example, a restaurantclosing time may not be displayed because the user has walked away fromthe restaurant.

In closing it is to be understood that the embodiments of the inventiondisclosed herein are illustrative of the principals of the invention.Other modifications may be employed which are within the scope of theinvention. Accordingly, the present invention is not limited to thatprecisely as is shown and described in the present specification.

We claim:
 1. A method comprising: providing content data to a portabledevice such that the content data is accumulated in a memory of theportable device, the portable device having a display screen; andproviding at least one time prompt to the portable device to trigger theportable device to retrieve a portion of the content data from thememory and display the portion of the content data on the display screenin synchronization with a display of a corresponding portion of a mediapresentation at a media presentation display device at the at least onetime prompt, the media presentation being distinct from the contentdata, the media presentation display device being distinct from theportable device, the portion of the content data being accumulated inthe memory prior to the display of the corresponding portion of themedia presentation.
 2. The method of claim 1, wherein the content datais accessed by the portable device through an interactive application.3. The method of claim 2 wherein the portable device has a touch screenthat receives an input for the interactive application.
 4. The method ofclaim 1, wherein the content data is provided through an IR signal. 5.The method of claim 1, wherein the at least one time prompt is providedthrough an IR signal.
 6. The method of claim 1, wherein the content isprovided from a first data source and the at least one time prompt isprovided from a second data source, the first data source is distinctfrom the second data source.
 7. The method of claim 1, wherein the mediapresentation is live.
 8. The method of claim 1, wherein the mediapresentation is prerecorded.
 9. The method of claim 1, wherein thecontent data is provided through an RF signal.
 10. The method of claim1, wherein the at least one time prompt is provided through an RFsignal.
 11. The method of claim 1, wherein the portable device is remotefrom the media display device.
 12. The method of claim 1, wherein theportable device has a wireless transmitter.
 13. The method of claim 1,wherein the portable device has a wireless receiver.
 14. The method ofclaim 1, wherein the memory is a cache memory.
 15. The method of claim1, wherein the portable device retrieves the portion of the content datafrom the memory by searching for the portion of the content data in thememory.
 16. A method comprising: providing content data to a portabledevice such that the content data is accumulated in a memory of theportable device and is accessible by an interactive application storedon the portable device, the portable device having a display screen, theportable device having a touch screen that receives an input for theinteractive application; and providing at least one time prompt to theportable device to trigger the portable device to retrieve a portion ofthe content data from the memory and display the portion of the contentdata on the display screen in synchronization with a display of acorresponding portion of a media presentation at a media presentationdisplay device at the at least one time prompt, the media presentationbeing distinct from the content data, the media presentation displaydevice being distinct from the portable device, the portion of thecontent data being accumulated in the memory prior to the display of thecorresponding portion of the media presentation.
 17. The method of claim16, wherein the content data is provided through an RF signal.
 18. Themethod of claim 16, wherein the at least one time prompt is providedthrough an RF signal.
 19. The method of claim 16, wherein the content isprovided from a first data source and the at least one time prompt isprovided from a second data source, the first data source is distinctfrom the second data source.
 20. The method of claim 16, wherein theportable device retrieves the portion of the content data from thememory by searching for the portion of the content data in the memory.21. A method comprising: providing content data from a first data sourceto a portable device via an RF signal such that the content data isaccumulated in a memory of the portable device and is accessible by aninteractive application stored on the portable device, the portabledevice having a display screen, the portable device having a touchscreen that receives an input for the interactive application; andproviding at least one time prompt to the portable device to trigger theportable device to retrieve a portion of the content data from thememory and display the portion of the content data on the display screenin synchronization with a display of a corresponding portion of a mediapresentation at a media presentation display device at the at least onetime prompt, the media presentation being distinct from the contentdata, the media presentation display device being distinct from theportable device, the portion of the content data being accumulated inthe memory prior to the display of the corresponding portion of themedia presentation, the media presentation being received from a seconddata source that is distinct from the first data source.